The present disclosure relates generally to image alignment and, in some embodiments, to a technique for aligning facial images.
Model-based image registration/alignment is a topic of interest in computer vision, where a model is deformed such that its distance to an image is minimized. In particular, face alignment is of interest as it enables various practical capabilities (e.g., facial feature detection, pose rectification, and face animation) and poses scientific challenges due to facial appearance variations in pose, illumination, expression, and occlusions. Previous techniques include the Active Shape Model (ASM), which fit a statistical shape model to an object class. ASM was extended to the Active Appearance Model (AAM), which has been used in face alignment. During AAM-based model fitting, the Mean-Square-Error between the appearance instance synthesized from the appearance model and the warped appearance from the input image is minimized by iteratively updating the shape and/or appearance parameters. Although AAM may perform reasonably well while learning and fitting on a small set of subjects, its performance degrades quickly when it is trained on a large dataset and/or fit to subjects that were not seen during the model learning.
In addition to the generative model based approaches such as AAM, there are also discriminative model based alignment approaches. The Boosted Appearance Model (BAM) utilizes the same shape model as AAM, but an entirely different appearance model that is essentially a two-class classifier and learned discriminatively from a set of correctly and incorrectly warped images. During model fitting, BAM aims to maximize the classifier score by updating the shape parameter along the gradient direction. Though BAM has shown to generalize better in fitting to unseen images compared to AAM, one potential issue is that the learned binary classifier cannot guarantee a concave score surface while perturbing the shape parameter. In other words, moving along the gradient direction does not always improve the alignment. The Boosted Ranking Model (BRM) alleviates this issue by enforcing the convexity through learning. Using pairs of warped images, where one is a better alignment than the other, BRM learns a score function that attempts to correctly rank the two warped images within all training pairs. While BRM may provide certain benefits over previous techniques, further improvements in image alignment may be achieved as described below.